1. Field
The subject invention is in the field of electronic processes which involve both hardware and software. Specifically, it is in the field of such processes related to production and control of facial representations such as on photographer's film, computer driven cathode ray tubes or computer driven robotic faces. More specifically, it is in the field of such processes based on real time electronic capture and representation of motions of all portions of human faces.
2. Prior Art
The documents listed below are a sampling of prior art related to computer generated and/or controlled animated representations of human faces, some operating in real time.
U.S. Patent Documents:
______________________________________ 5,500,671 3/19/96 Andersson et al. 5,479,529 12/26/95 Nakagawa et al. 5,347,306 9/13/94 Nitta 5,009,626 4/23/91 Katz 4,827,532 5/2/89 Bloomstein 5,129,014 7/7/92 Bloomberg ______________________________________
Non-Patent Documents:
Terzopoulos et al., Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models, IEEE Transactions on Pattern Analysis, v. 14, n. 6, June 1993, p. 569-579.
Ohya et al., Real-Time Reproduction of 3D Human Images in Virtual Space Teleconferencing, IEEE Virtual Reality Annual Symposium, Sep. 22, 1993, p. 408-414.
Choi et al., Analysis and Synthesis of Facial Image Sequences in Model Based Image Coding, IEEE Transactions on Circuits and Systems for Video Technology, June 1994,p. 257-275.
Singh et al., Control and Coordination of Head, Eyes and Facial Expressions of Virtual Actors in Virtual Environments, IEEE Robot and Human Communication, 1993 Int'l Workshop, 11/93, p. 335-339.
Several techniques are shown for (1) detecting and recording the motions and deflections of portions of a human face in real time as expressions are generated and changing and (2) processing and using the recorded information to produce facial images showing the recorded expressions or to drive the actuation of a robotic face model, of a human face or not. However, it is known that there is demand for techniques for the purposes described which provide even more life-like representations and are less costly to use, partly because less time and effort are involved in producing better quality results.
Limitations of prior art systems for detecting and recording (capturing) the motions, deflections and related characteristics of a human face are: (1) the equipment and techniques do not provide enough detail for producing discrete realistic expressions in the representations; (2) available computers have not been able to allow prior art systems to work at a real-time standard rate of 15 to 30 frames per second; (3) the captured human facial input does not properly create or control faces which are significantly different from the face from which the input data was captured such as, in particular non-human faces; (4) facial features such as wrinkles and skin stretching and sliding across a face have not been managed to a high enough degree of quality and detail; and (5) the facial input has not been sufficiently successfully transmuted to the output representation.
Accordingly, the primary objective of the subject invention is to provide a process using known facial expression capture techniques and overcoming the stated limitations. Further objectives are that the process accommodate any face for input purposes in real time function and produce fluidly animated representations of humans and non-humans with detail sufficient to show subtle changes in appearances such as glowers and scowls. A further objective is that the subject process produce significantly superior products at significantly less cost.